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ABSTRACT 


We  introduce  a  new  surface  representation  for  recognizing  curved  objects.  Our  approach 
begins  by  representing  an  object  by  a  discrete  mesh  of  points  built  from  range  data  or  from 
a  geometric  model  of  the  object  The  mesh  is  computed  from  the  data  by  deforming  a  stan¬ 
dard  shaped  mesh,  for  example,  an  ellipsoid,  until  it  fits  the  surface  of  the  object.  We 
define  local  regularity  constraints  that  the  mesh  must  satisfy.  We  then  define  a  canonical 
mapping  between  the  mesh  describing  the  object  and  a  standard  spherical  mesh.  A  surface 
curvature  index  that  is  pose-invariant  is  stored  at  every  node  of  the  mesh.  We  use  this 
object  representation  for  recognition  by  comparing  the  spherical  model  of  a  reference 
object  with  the  model  extracted  from  a  new  observed  scene.  We  show  how  the  similarity 
between  reference  model  and  observed  data  can  be  evaluated  and  we  show  how  the  pose 
of  the  reference  object  in  the  observed  scene  can  be  easily  computed  using  this  representa¬ 
tion. 

We  present  results  on  real  range  images  which  show  that  this  approach  to  modelling  and 
recognizing  three-dimensional  objects  has  three  main  advantages:  First,  it  is  applicable  to 
complex  curved  surfaces  that  cannot  be  handled  by  conventional  techniques.  Second,  it 
reduces  the  recognition  problem  to  the  computation  of  similarity  between  spherical  distri¬ 
butions;  in  particular,  the  recognition  algorithm  does  not  require  any  combinatorial  search. 
Finally,  even  though  it  is  based  on  a  spherical  mapping,  the  approach  can  handle  occlu¬ 
sions  and  partial  views. 
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1  Introduction 


Recognition  of  curved  objects  is  one  of  the  key  issues  in  computer  vision.  It  is  a  problem 
not  (Hily  in  traditional  applications  such  as  industrial  object  recognition  and  face  recogni- 
titm  but  also  in  emerging  applications  such  as  navigation  and  manipulation  in  natural  envi- 
rcHunents.  To  aid  in  overcoming  this  problem,  we  have  designed  a  new  s^proach  that  uses 
as  a  starting  point  a  combination  of  several  traditional  object  recognition  and  representa¬ 
tion  methods. 

Traditionally,  there  are  two  ways  to  represent  objects  for  recognition;  local  and  global. 
Local  methods  attempt  to  represent  obj^ts  as  a  set  of  primitives  such  as  faces  or  edges. 
Most  eariy  local  methods  handle  polyhedral  objects  and  report  effective  and  encouraging 
results.  Representative  systems  include  [12][201[14].  Few  systems  can  handle  curved  sur¬ 
faces.  Some  systems  include  early  work  in  which  primitive  surfaces  enclosed  by  orienta¬ 
tion  discontinuity  boundaries  are  extracted  from  range  data  [21].  Other  systems  determine 
primitive  surfaces  which  satisfy  planar  or  quadric  equations  [9].  Techniques  based  on  dif¬ 
ferential  geometry  such  as  [3]  segment  range  images  using  Gaussian  curvatures.  More 
recent  local  techniques  use  points  of  intoest  and  edges  of  surfaces  to  match  observed  sur¬ 
faces  with  stored  representation  [23].  These  local  methods,  however,  are  noise-sensitive 
and  are  still  limited  in  reliably  extracting  primitives  of  curved  objects  from  input  images. 

The  global  methods  assume  one  particular  coordinate  system  attached  to  an  object  and 
represent  the  object  as  an  implicit  or  parametric  function  in  this  coordinate  system.  The 
resulting  representation  is  global  in  that  the  implicit  function  represents  the  entire  shape  of 
the  object  or  of  a  large  portion  of  the  object  TTie  generalized  cylinder  (GC)  is  a  represen¬ 
tative  of  this  group.  A  generalized  cylinder  is  defined  as  an  axis,  a  cross-sectional  shape 
and  its  sweeping  rule  along  the  axis.  Although  encouraging  results  have  been  obtained  in 
recognizing  GCs  in  intensity  images  by  minimizing  the  distance  between  observed  and 
predicted  occluding  edges,  using  generalized  cylinders  for  recognition  is  difficult  due  to 
the  difficulty  of  extracting  GC  parameters  from  input  images. 

Superquadrics  (SQ)  representation  also  belongs  to  the  class  of  global  representations 
[22].  Superquadrics  are  generalizations  of  ellipsoids.  Object  representations  are  built  by 
fitting  an  implicit  equation  to  a  set  of  input  data  points.  Recognition  using  SQs  proceeds 
by  OKnparing  the  parameters  of  the  SQs  extracted  from  the  scene  with  the  SQs  stored  in 
the  model.  The  SQs  represent  a  limited  set  of  shapes  which  can  be  extended  by  adding 
parameters  to  the  generic  implicit  equation  of  SQs.  This  limitation  has  the  undesirable 
effect  of  making  the  fitting  process  much  more  expensive  and  numerically  unstable.  A 
possible  extension  is  to  segment  objects  into  sets  of  superquadrics  [10],  although  the  com- 
putaticMial  complexity  of  the  scene  analysis  may  become  prohibitive.  An  interesting 
attempt  to  handle  a  large  class  of  natural  objects  in  discussed  in  [4]  in  which  multiple  sur¬ 
face  representations,  ranging  from  quadrics  to  superquadrics  to  generalized  cylinders,  are 
used,  llie  type  of  representation  is  selected  based  on  the  level  of  detail  available  from  the 
range  image. 

EGI  and  CEGI  map  surface  orientation  distributions  to  the  Gaussian  sphere  [13][18][1S]. 
Since  the  Gauss  map  is  independent  on  translation,  the  representation  is  quite  suitable  to 
handle  convex  curved  objects.  In  this  case,  recognition  proceeds  by  findhig  the  rotation 
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that  maximizes  the  correlation  between  two  EGIs  [13][7].  However,  when  part  of  the 
object  is  occluded,  those  techniques  cannot  reliably  extract  the  representation. 

Recently,  new  approaches  for  modeling  objects  have  been  developed.  These  approaches 
are  based  on  the  idea  of  fitting  a  bounded  algebraic  surface  of  fixed  degree  to  a  set  of  data 
points  [24][25].  Wfith  this  representation,  recognition  proceeds  by  comparing  the  polyno¬ 
mials  describing  observed  and  stored  surfaces,  although  it  is  not  yet  clear  how  the  compar¬ 
ison  would  be  performed.  Using  algebraic  surfaces  is  convenient  because  powerful  tools 
can  be  used  to  confute  limbs  and  other  properties  of  the  object  Recognition  proceeds  by 
comparing  invariant  properties  computoi  from  the  algebraic  equations  of  observed  and 
reference  surfaces  [11].  Although  encouraging  results  have  been  obtained  in  this  area, 
more  research  is  needed  in  the  areas  of  bounding  constraints,  convergence  of  surface  fit¬ 
ting,  and  recognition  before  this  approach  becomes  practical.  Occlusion  remains  a  prob¬ 
lem  since  there  is  no  guarantee  that  the  polynomial  computed  from  a  partial  view  is 
similar  to  the  polynomial  computed  from  a  complete  model  of  the  object.  For  a  survey  of 
other  techniques  can  be  used  for  global  surface  fitting,  see  [S]. 

All  these  approaches  attempt  to  fit  some  known  parametric  surface,  either  locally  or  glo¬ 
bally,  to  the  object  Consequently,  these  approaches  tend  to  limit  the  set  of  shapes  that  can 
be  represented  and  recognized.  Moreover,  the  cost  of  building  the  representations  from 
data  sets  increases  rapidly  as  parameters  are  added  to  expand  the  set  of  allowable  shapes. 
To  address  these  two  problems,  another  class  of  approaches  attempts  to  match  sets  of 
points  directly  without  any  prior  surface  fitting.  An  example  is  the  work  by  Besl  [2]  in 
which  the  distance  between  point  sets  is  computed  and  minimized  to  find  the  best  transfor¬ 
mation  between  model  and  scene.  This  approach  has  many  advantages  since  it  does  not 
requite  any  surface  segmentation  or  surface  fitting  and  it  does  not  require  to  search  for  an 
explicit  correspondence  between  model  and  scene  features  for  recognition.  Recent  results 
show  that  these  algorithms  can  perform  remarkably  well  by  using  new  numerical  tech¬ 
niques  for  miiumizing  distances  between  two  arbitrary  point  sets.  The  main  drawback  of 
this  approach  is  that,  like  any  minimization  techiuque,  it  is  not  guaranteed  to  find  the  glo¬ 
bal  optimum  especially  if  the  scene  contains  occlusions,  different  point  density  in  model 
and  scene  representation,  and  large  number  of  extra  points  frrom  different  objects. 

Our  approach  begins  with  a  combination  of  the  point  set  matching  and  of  the  original 
EGI  approach.  As  in  the  case  of  the  point  set  matching,  we  want  to  avoid  fitting  analytical 
surfaces  to  represent  an  object  Instead,  we  use  a  representation  that  simply  consists  of  a 
collection  of  points,  or  nodes,  arranged  in  a  mesh  covering  the  entire  surface  of  the  object. 
This  has  the  advantage  that  the  object  can  have  any  arbitrary  shape,  as  long  as  that  shape  is 
topologically  equivalent  to  the  sphere.  To  avoid  problems  with  variable  density  of  nodes 
on  the  mesh,  we  need  to  define  regularity  constraints  that  must  be  enforced  when  the  mesh 
is  built  Constructing  meshes  that  fit  input  data  and  that  satisfy  some  constraints  is  possible 
based  on  the  optimization  techniques  originally  introduced  in  [26]  and  [16].  We  use  an 
extension  of  the  deformable  surfaces  algorithms  introduced  in  [8]  to  compute  the  meshes. 
As  in  the  EGI  algorithms,  each  node  of  the  mesh  is  mapped  onto  a  tegular  mesh  on  the 
unit  sphere,  and  a  quantity  that  reflects  the  local  surface  curvature  at  the  node  is  stored  at 
the  corresponding  node  on  the  sphere.  Instead  of  using  a  discrete  approximation  of  the 
curvature,  we  develop  a  new  curvature  indicator,  the  simplex  angle,  which  is  entirely 
defined  fixmi  a  node  and  its  neighbors  in  the  mesh  without  any  reference  to  the  underlying 


continuous  surface.  We  call  the  corresponding  spherical  representation  the  Simplex  Angle 
Image  (SAI).  Finally,  we  define  the  regularity  constraints  such  that  if  fxf  is  the  mesh  repre¬ 
senting  an  object,  and  lM"is  the  mesh  representing  the  same  object  after  transformation  by 
a  combination  of  rotation,  translation,  and  scaling,  then  the  corresponding  distributions  of 
simplex  angles  on  the  spherical  representations  S  and  S'  are  the  same  up  to  a  rotation.  In 
other  words,  the  SAI  is  an  invariant  representation.  Therefore,  to  determine  whether  two 
objects  are  the  same,  we  need  only  compare  the  corresponding  spherical  distributions.  The 
overall  approach  is  illustrated  in  Figure  1:  A  regular  mesh  is  computed  from  input  object 
description,  sensor  data  or  CAD  model,  simplex  angle  is  conq)uted  at  each  node  of  the 
meshes  and  the  meshes  are  mapped  onto  a  sphere,  the  SAI.  If  a  rotation  between  the  two 
spherical  images  exists,  the  two  meshes  correspond  to  the  same  object  This  approach  is 
similar  in  principle  to  the  EGI  approach.  However,  one  ftmdamental  difference  is  that  a 
unique  mesh,  up  to  rotation,  translation  and  scale,  can  be  reconstructed  from  a  given  SAI. 
In  the  case  of  the  EGI,  this  property  is  true  only  for  convex  objects.  Another  fundamental 
difference  is  that  the  SAI  preserves  connectivity  in  that  patches  that  are  connected  on  the 
surface  of  the  input  object  are  still  cormected  in  the  spherical  representation.  The  latter  is 
the  main  reason  why  our  approach  can  handle  arbitrary  non-convex  objects.  Connectivity 
conservation  is  also  the  reason  why  the  SAI  can  be  used  fcH*  recognition  even  in  the  pres¬ 
ence  of  significant  occlusion,  as  we  will  see  later  in  the  paper,  whereas  EGI  and  other  glo¬ 
bal  representation  cannot 


Figure  1  Object  Recognition  Using  SAIs 

Another  way  to  describe  the  properties  of  the  SAI  is  in  terms  of  intrinsic  coordinate  sys¬ 
tems.  An  intrinsic  coordinate  is  such  that  any  given  surface  point  has  the  same  coordinates 
regardless  of  the  orientation,  position,  and  scale  of  the  underlying  object  In  the  case  of 
two-dimensional  contours,  intrinsic  coordinate  systems  are  very  easy  to  define.  For  exam¬ 
ple,  they  are  the  basis  of  geometric  hashing  techniques  [17].  For  three-dimensional  sur¬ 
faces,  a  general  definition  of  intrinsic  coordinate  systems  on  curved  objects  is  much  more 
difficult  to  define.  For  example,  the  geodesics  of  a  surface  can  be  used  to  define  an  intrin¬ 
sic  coordinate  system.  Still  other  efforts  focus  on  lines  of  curvatures  and  other  differential 
geometry  invariants  [6].  The  problem  with  these  approaches  is  that  they  are  based  on  defi- 
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nitions  and  properties  that  are  valid  for  continuous  surfaces,  whereas  we  typically  have  to 
handle  discrete  surfaces  from  sensor  data. 

To  describe  our  approach,  we  have  organized  the  paper  as  follows.  In  Section  2,  we 
describe  a  simple  representation  of  closed  2-D  curves  which  then  extend  to  three  dimen¬ 
sional  surfaces  in  Section  3.  In  both  sections,  we  investigate  the  notions  of  global  and 
local  regularity,  and  of  simplex  angle.  The  two-dimensional  representation  is  fairly  stan¬ 
dard  and  is  used  here  only  as  introduction  to  the  definition  of  the  equivalent  representation 
in  three  dimensions.  In  Section  4,  we  show  how  to  obtain  SAIs  fi*om  range  data.  In  Section 
S,  we  describe  the  SAI  matclung.  In  Sections  3  to  5  we  describe  the  fundamentals  of  the 
SAI  algorithms  in  the  case  of  complete  object  models.  We  address  the  problem  of  occlu¬ 
sion  and  partial  models  in  Section  6.  In  that  section,  we  also  present  several  results  of  rec- 
ogiution  in  complex  scenes  and  a  discussion  of  performance  and  robustness  of  the 
recognition  algorithm. 


2  Intrinsic  Representation  of  2-D  Curves 

A  standard  approach  to  representing  and  recognizing  contours  is  to  approximate  con¬ 
tours  by  polygons,  and  to  compute  a  quantity  that  is  related  to  the  curvature  of  the  under¬ 
lying  curve.  The  similarity  between  contours  can  then  be  evaluated  by  comparing  the 
distribution  of  curvature  measurement  at  the  vertices  of  the  polygonal  representations. 
Under  certain  conditions,  the  curvature  distribution  can  be  mapped  unambiguously  on  the 
unit  circle,  allowing  for  a  representation  that  is  independent  of  orientation  and  position  of 
the  contour.  In  this  section,  we  introduce  the  basic  concepts  that  can  be  used  to  manipulate 
polygonal  representations  of  contours.  Starting  with  the  definition  of  a  curvature  indicator, 
we  then  define  conditions  of  local  and  global  regularity  for  building  intrinsic  representa¬ 
tions  of  contours.  The  concepts  discussed  in  this  section  are  well  known  and  have  been 
studied  and  applied  extensively  in  previous  works.  Our  purpose  here  is  to  introduce  them 
in  a  way  that  facilitates  their  extension  to  three-dimensional  surfaces.  In  particular,  the  cir¬ 
cular  mapping  is  equivalent  to  the  classical  9-5  representation. 

Rather  than  attempting  to  approximate  the  curvature  of  a  discrete  curve  at  each  node  of 
the  polygonal  approximation,  we  use  a  different  quantity,  the  angle  9  between  consecutive 
segments  (Figure  2),  which  is  related  to  the  curvature  but  has  more  desirable  properties  in 
dealing  with  discrete  representations.  The  relation  between  9  and  the  curvature  k  is 

k»<p/l  as  I  becomes  small,  or,  equivalently,  as  the  density  of  points  increases.  Like 
the  curvature,  the  angle  9  is  independent  of  rotation  and  translation.  Unlike  the  curvature, 
9  is  also  independent  of  scaling. 
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Figure  2  Discrete  Curvature  Measure  of  a  2-D  Curve 

One  problem  is  that  if  the  lengths  of  the  segments  representing  the  curve  are  allowed  to 
vary,  the  value  of  <p  depends  not  only  on  the  shape  of  the  curve  but  also  on  the  distribution 
of  points  on  the  curve.  In  particular,  it  is  important  for  the  same  curve  shape  to  generate 
the  same  value  of  (p  to  unable  the  comparison  of  discrete  curves.  One  way  to  avoid  this 
problem  is  to  impose  a  local  regularity  condition  on  the  distribution  of  vertices.  The  local 
regularity  condition  simply  states  that  all  the  segments  must  have  the  same  length.  This 
can  be  restated  as  a  local  property  by  saying  that  the  length  of  a  segment  must  be  equal  to 
the  length  of  each  neighboring  segments.  Another  geometric  definition  of  this  condition  is 
illustrated  in  Hgure  3.  The  condition  that  the  length  of  the  two  segments  PPj  and  PP2  are 
the  same  is  equivalent  to  the  condition  that  the  projection  of  node  P  on  the  line  joining  its 
two  neighbors  Pi  and  P2  coincides  with  the  center  of  Pj  and  P2.  This  is  obviously  a  more 
complicated  way  of  formulating  the  simple  regularity  condition,  but  it  will  become  useful 
when  we  extend  this  notion  to  three  dimensions.  One  consequence  of  the  regularity  condi¬ 
tion  is  that  there  is  only  one  degree  of  freedom  in  the  polygonal  representation  in  that,  for 
a  given  number  of  vertices,  specifying  one  vertex  on  the  curve  determines  uniquely  the 
locations  of  all  the  other  nodes  along  the  curve. 

P 
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Figure  3  Local  Regularity 

The  last  step  in  representing  two-dimensional  contours  is  to  build  a  circular  representa¬ 
tion  that  can  be  used  for  recognizing  contours.  Let  us  assume  that  the  contour  is  divided 
into  N  segments  with  vertices  Pi,.J*fi,  and  with  corresponding  angles  97,.,<p/v  “s 
divide  the  unit  circle  using  N  equally  spaced  vertices  Finally,  let  us  store  the 

angle  9,-  associated  with  P^  at  the  corresponding  circle  point  Cj  (Figure  4).  The  circular 
representation  of  the  contour  is  invariant  by  rotation,  translation,  and  scaling.  In  other 
words,  given  the  position  of  two  vertices,  a  circular  representation  defines  a  unique  polyg¬ 
onal  contour.  The  two  vertices  are  necessary  to  define  the  rotation,  translation,  and  scaling. 
Gmversely,  as  the  density  of  points  becomes  large,  the  circular  representations  of  two 
instances  of  the  same  contour  are  identical  up  to  a  rotation  of  the  unit  sphere.  This  prop¬ 
erty  allows  for  comparing  contours  by  deciding  that  two  contours  are  identical  if  there 
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exists  a  rotation  of  the  unit  circle  that  brings  their  representation  in  correspondence  (Fig¬ 
ure  5).  The  unicity  property  is  true  because  of  the  local  regularity  condition  and  because  of 
the  invariance  of  <p.  Also,  when  comparing  contours,  the  distribution  of  the  vertices  C,  on 
the  circle  must  be  uniform,  that  is  the  distance  between  consecutive  vertices  on  the  unit 
circle  is  constant  We  refer  to  this  property  as  global  regularity.  Global  regularity  can 
always  be  achieved  in  the  case  of  planar  curves.  We  mention  it  here  because  it  becomes  a 
non-trivial  notion  when  we  try  to  extend  the  discrete  representation  to  3-D  surfaces. 


Figure  4  Mapping  Between  Shape  and  Circular  Representation  Space 


rotation  of  unit  circle 


contours 


‘Ps 

circular  representations 


Figure  5  Comparing  Contours  in  Representation  Space 


3  Intrinsic  Representation  of  3-D  Surfaces 

In  this  section  we  extend  the  concepts  of  curvature  indicator,  local  and  global  regularity, 
and  circular  representation  to  three-dimensional  surfaces.  We  consider  the  case  of  repre¬ 
senting  surfaces  topologically  equivalent  to  the  sphere.  (Cases  in  which  only  part  of  the 
surface  is  visible  will  be  addressed  in  Section  6.)  We  follow  the  same  approach  as  for  two- 
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dimensional  contours.  We  first  define  a  discrete  representation  of  surfaces,  the  equivalent 
of  the  polygonal  representation,  in  Section  3.1.  We  introduce  the  three-dimensional  equiv¬ 
alent  of  the  concepts  of  global  and  local  regularity  in  Sections  3.2  and  3.3,  respectively.  In 
Section  3.4,  we  propose  a  new  indicator  of  curvature,  the  simplex  angle,  that  is  a  direct 
extension  of  the  angle  used  in  the  two-dimensional  case.  Finally,  we  define  an  intrinsic 
spherical  representation  as  an  extension  of  the  circular  representation  of  contours  in  Sec¬ 
tion  3.5.  At  the  end  of  this  section,  we  will  have  defined  a  representation  that  is  invariant 
by  translation  and  scale  and  is  unique  for  a  given  object  and  a  given  resolution  up  to  a 
rotation  of  the  representation  space.  Detailed  presentations  of  the  basic  results  on  semi¬ 
regular  tessellations,  triangulations,  and  duality  can  be  found  in  [19][27][28]. 

3.1  TViangulation  and  Duality 

The  most  natural  discrete  representation  of  a  surface  is  a  triangulation,  that  is  a  polyhe¬ 
dron  with  triangular  faces  whose  vertices  are  on  the  surface.  Each  face  defines  a  plane 
which  is  the  local  approximation  of  the  surface.  It  is  desirable  for  many  algorithms  to  have 
a  constant  number  of  neighbors  at  each  node.  We  use  a  class  of  meshes  such  that  each 
node  has  exactly  three  neighbors.  Such  meshes  can  always  be  constmcted  as  the  dual  of  a 
triangulation.  The  dual  of  a  triangulation  is  a  graph  with  one  node  for  each  face  of  a  trian¬ 
gulation.  Nodes  are  connected  in  the  dual  graph  if  they  correspond  to  connected  faces  in 
the  original  triangulation.  Figure  6  shows  a  triangulation  and  its  dual.  In  switching  from  a 
trianguladon  to  its  dual,  the  property  of  planarity  of  the  faces  is  lost  since  faces,  defined  as 
the  cycles  with  the  minimum  number  of  vertices,  may  have  more  than  three  vertices. 
Therefore,  the  dual  mesh  should  be  viewed  as  a  graph  of  points  with  the  desired  connec¬ 
tivity;  the  trianguladon  may  be  viewed  as  a  polyhedral  approximation  of  the  object.  The 
dual  of  any  trianguladon  is  a  graph  of  degree  three.  In  the  remainder  of  this  paper,  we  will 
use  only  dual  meshes  with  the  understanding  that  they  can  be  derived  from  an  initial  trian¬ 
gulation. 


Figure  6  IViangulation  and  Dual  Mesh 

3.2  Global  Regularity  and  Mesh  Topology 

As  mentioned  in  the  previous  section,  global  regularity  can  be  easily  achieved  in  two 
dimensions  since  a  curve  can  be  always  divided  into  an  arbitrary  number  of  segments  of 
equal  length.  The  equivalent  in  three  dimensions  would  be  a  mesh  covering  a  closed  sur¬ 
face  such  that  the  distance  between  vertices  is  constant  and  is  the  dual  of  a  trianguladon. 
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that  is,  each  node  has  exactly  three  neighbors.  Extending  the  notion  of  global  regularity  to 
a  mesh  covering  a  two  dimensional  plane,  there  are  three  possible  topologies:  The  triangu¬ 
lar  and  hexagonal  meshes  which  are  dual  of  each  other,  and  the  square  mesh  which  is  its 
own  dual.  The  problem  is  that,  even  though  these  meshes  provide  global  regularity  for  an 
open  surface,  they  cannot  be  extended  to  a  close  siuface.  In  fact,  tetrahedron,  cube  and 
dodecahedron  (Figure  7). are  the  only  regular  triangulation-dual  tessellations  of  a  closed 
surface,  corresponding  to  the  triangular,  square,  and  hexagonal  topologies,  respectively. 
Therefore,  only  approximate  global  regularity  can  be  achieved  in  three  dimensions. 

The  approach  that  we  use  is  recursive  subdivision  of  the  dodecahedron  which  yields  a 
mesh  that  is  “almost”  regular  in  that  all  but  12  pentagonal  cells  have  hexagonal  connectiv¬ 
ity.  If  the  number  of  cells  is  large  enough,  typically  several  hundred  to  a  few  thousand,  the 
ratio  of  the  number  of  regular  hexagonal  cells  to  the  number  of  singular  pentagonal  cells 
becomes  very  small.  Therefore,  the  mesh  is  almost  regular  for  a  large  number  of  cells.  In 
practice,  a  triangulation  with  the  appropriate  number  of  nodes  is  first  constructed.  The  tri¬ 
angulation  is  built  by  subdividing  each  triangular  face  of  a  20-face  icosahedron  into 
smaller  triangles.  The  final  mesh  is  built  by  taking  the  dual  of  the  2(W^  faces  triangulation, 
yielding  a  mesh  with  the  same  number  of  nodes.  Figure  8  shows  the  mesh  obtained  by 
recursive  subdivision  of  the  dodecahedron  for  N  -2,  3,  and  5.  For  the  experiments  pre¬ 
sented  in  this  paper,  we  used  a  subdivision  frequency  of  ^  =  7  for  a  total  number  of  nodes 
of  980. 


Figure  7  Regular  Meshes 


(a)  N  =  2  (b)  N  =  3  (c)  N  =  5 

Figure  8  Meshes  firom  Recursive  Subdivision  of  Dodecahedron 
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33  Local  Regularity 

The  next  step  in  going  from  two  to  three  dimensions  is  to  define  a  notion  of  local  regular¬ 
ity  that  leads  to  invariance  properties  of  the  mesh  and  curvature  indicator  definition  simi¬ 
lar  to  the  properties  used  for  2-D  curves.  The  definition  of  local  regularity  in  three 
dimensions  is  a  straightforward  extension  of  the  definition  of  Section  2.  Let  P  be  a  node  of 
the  mesh,  P/,  P2,  Pj  be  its  three  neighbors,  G  be  the  centroid  of  the  three  points,  and  Q  be 
the  projection  of  P  on  the  plane  defined  by  Pj,  P2,  and  P^  (Figure  9).  The  local  regularity 
condition  simply  states  that  Q  coincides  with  G.  This  is  the  same  condition  as  in  two 
dimensions,  replacing  the  triangle  (Pj,  P2,  P)  of  Figiire  3  by  the  tetrahedron  (Pj,  P2,  P3, 
P).  The  local  regularity  condition  is  invariant  by  rotation,  translation,  and  scaling  because 
it  is  purely  local  and  involves  only  relative  positions  of  the  nodes  with  respect  to  each 
other,  not  absolute  distances. 


Q  projection  of  P 
on  (PiPzPs) 


Figure  9  Local  Regularity  in  Three  Dimensions 


3.4  Discrete  Curvature  Measure:  Simplex  Angle 

The  last  step  in  building  a  discrete  surface  representation  is  to  define  an  indicator  of  cur¬ 
vature  that  can  be  computed  from  a  mesh  with  the  appropriate  regularity  properties.  We 
propose  a  definition  in  terms  of  angular  variation  between  neighbors  in  the  mesh  accord¬ 
ing  to  the  definition  of  Figure  2.  We  need  to  define  some  notations  (Figme  10  (a)).  Let  P 
be  a. node  of  the  mesh,  P;,  P2,  Pj  its  three  neighbors,  O  the  center  of  the  sphere  circum¬ 
scribed  to  the  tetrahedron  (P,  P;,  P2,  Pj),  Z  the  line  passing  through  O  and  through  the 
center  of  the  circle  circumscribe  to  (PiJP2J*3)'  Now,  let  us  consider  the  cross  section  of 
the  surface  by  the  plane  11  containing  Z  and  P.  The  intersection  of  11  with  the  tetrahedron 
is  a  triangle.  One  vertex  of  the  triangle  is  P,  and  the  base  opposite  to  P  is  in  the  plane 
(P]J*2^3)  (Figure  10  (b)).  We  define  the  angle  9^  as  the  angle  between  the  two  edges  of 
the  triangle  intersecting  at  P.  By  definition,  9^  is  the  discrete  curvature  measure  at  node  P. 
It  is  easy  to  see  that  this  definition  is  consistent  with  the  2-D  definition  since  the  geometry 
in  the  plane  n  is  the  same  as  the  initial  geometry  for  a  two  dimensional  curve  illustrated  in 
Figure  2.  We  call  9^  the  simplex  angle  at  P,  since  it  is  the  extension  to  a  three-dimensional 
simplex,  the  tetrah^on,  of  the  notion  introduced  for  a  two-dimensional  simplex,  the  tri¬ 
angle. 
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(a)  (•>) 

Figure  10  Definition  of  the  Simplex  Angle 


The  simplex  angle  varies  between  -x  and  n.  The  angle  is  0  for  a  flat  surface,  and  is  large 
in  absolute  value  if  i*  is  far  from  the  plane  of  its  three  neighbors.  The  simplex  angle  is  neg¬ 
ative  if  the  surface  is  locally  concave,  positive  if  it  is  convex,  assuming  that  the  set  of 
neighbors  is  oriented  such  that  the  normal  to  the  plane  they  form  is  pointing  toward  the 
outside  of  the  object  (Figure  11).  This  behavior  of  the  simplex  angle  corresponds  to  the 
intuitive  notion  of  local  “curvature”  of  a  surface.  Another  desirable  property  is  that  the 
simplex  angle  is  sphere-invariant  in  that  remains  the  same  no  matter  where  P  is  located 
on  the  circumscribed  sphere.  In  particular,  this  implies  that  if  the  nodes  of  the  mesh  are  on 
a  surface  whose  curvature  is  constant  in  a  region,  then  the  simplex  angle  will  also  be  con¬ 
stant  in  this  region  no  matter  what  the  distribution  of  the  points  is.  Finally,  it  is  clear  that 
the  simplex  angle  is  invariant  by  rotation,  translation,  and  scaling. 


(a)  Flat  plane:  iPo  =  0  (b)  Convex  peak:  close  to  n  (c)  Concave  peak:  9o  close  to  -x 


Figure  11  Typical  Values  of  the  Simplex  Angle 
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There  are  two  practical  problems  with  this  definition  of  the  simplex  angle.  First,  this  def¬ 
inition  requires  the  computation  of  a  circumscribed  circle  and  of  a  circumscribed  sphere, 
both  of  which  are  expensive  operations.  This  computation  becomes  a  problem  in  the  algo- 
ritiun  of  Section  4  in  which  the  angle  has  to  be  evaluated  many  times  at  every  node  of  a 
1000-node  mesh.  Second,  the  radius  of  the  circumscribed  sphere  becomes  infinite  as  9^ 
vanishes.  As  a  result,  a  direct  implementation  using  the  radius  of  the  circumscribed  sphere 
is  expected  to  be  numerically  unstable  in  areas  in  which  the  mesh  is  nearly  flat. 

In  contrast,  we  use  a  different  approach  that  leads  to  an  efficient  and  stable  way  to  com¬ 
pute  9^.  The  details  of  the  algorithm  are  given  in  Appendix  A. 

In  the  rest  of  the  paper,  we  will  denote  by  g  the  function  that  maps  a  node  to  its  simplex 
angle;  the  simplex  angle  9^  at  a  node  P  will  be  denoted  by  g(P). 

3.5  Simplex  Angle  Image 

We  have  extended  the  notions  of  regularity  and  simplex  angle  to  three-dimensional  sur¬ 
faces;  we  can  now  extend  the  circular  representation  developed  in  two  dimensions  to  a 
spherical  representation  in  three  dimensions.  Let  i\(he  mesh  of  points  on  a  surface  such 
that  it  has  the  topology  of  the  quasi-regular  mesh  of  Section  3.2.  Let  5  be  a  reference  mesh 
with  the  same  number  of  nodes  on  the  sphere  of  unit  one.  We  can  establish  a  one-to-one 
mapping  h  between  the  nodes  of  Afand  the  nodes  of  5.  The  mapping  h  depends  only  on  the 
topology  of  the  mesh  and  the  number  of  nodes.  Specifically,  for  a  given  size  of  the  mesh  M 
=  20xAr,  where  N  is  the  frequency  of  the  mesh  (Section  3.2),  we  can  define  a  canonical 
numbering  of  the  nodes  that  represents  the  topology  of  any  Af-mesh.  In  other  words,  if  two 
nodes  from  two  different  M-mesh  have  the  same  index,  so  do  their  neighbors.  With  this 
indexing  system,  h(JP\  where  F  is  a  node  of  the  spherical  mesh,  is  the  node  of  the  object 
mesh  that  has  correspond  to  the  same  index  as  P. 

In  the  current  implementation,  the  nodes  are  stored  in  a  two-dimensional  array.  The  first 
dimension  of  the  array,  sometimes  called  the  major  index,  is  between  1  and  20,  the  second 
dimension,  sometimes  called  the  nunor  index,  is  between  1  and  N^.  For  any  node  Pima- 
jor/ninor),  h(P)  is  the  point  stored  at  the  same  location,  (major/nittor),  in  the  object  mesh 
table.  The  connectivity  table  is  computed  only  once  for  any  given  frequency. 

Given  h,  we  can  store  at  each  node  F  of  5  the  simplex  angle  of  the  corresponding  node  on 
the  surface  g(/>(F)).  The  resulting  structure  is  a  quasi-regular  mesh  on  the  unit  sphere,  each 
node  being  associated  with  a  value  corresponding  to  the  simplex  angle  of  a  point  on  the 
original  surface.  By  analogy  with  the  Extended  Gaussian  Image,  we  call  this  representa¬ 
tion  the  Simplex  Angle  Image  (SAI).  In  the  remainder  of  the  paper,  we  will  denote  by  g(F) 
instead  of  gih(P))  the  simplex  angle  associated  with  the  object  mesh  node  h(P)  since  there 
is  no  ambiguity. 

If  the  original  mesh  satisfies  the  condition  of  local  regularity,  then  the  corresponding 
SAI  has  several  important  properties.  First,  the  SAI  is  invariant  by  translation  and  scaling 
of  the  original  object,  given  a  mesh  M.  This  condition  is  because  the  simplex  angle  itself  is 
invariant  by  translation  and  scaling  (Section  3.5),  and  because  lAf  still  satisfies  the  local 
regularity  condition  after  translation  and  scaling  (Section  3.3). 
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The  fundamental  property  of  the  S AI  is  that  it  represents  an  object  unambiguously  up  to 
a  rotation.  More  precisely,  if  and  are  two  meshes  on  the  same  object  with  the  same 
number  of  nodes  both  satisfying  the  local  regularity  condition,  then  the  corresponding 
SAIs  5and  5 'are  identical  up  to  a  rotation  of  the  unit  sphere.  Strictly  speaking,  this  is  true 
only  as  the  number  of  nodes  becomes  very  large  because  the  nodes  of  one  sphere  do  not 
necessarily  coincide  with  the  nodes  of  the  rotation  version  of  the  other  sphere.  (This  prob¬ 
lem  is  addressed  in  Section  S.l.)  One  consequence  of  this  property  is  that  two  SAIs  repre¬ 
sent  the  same  object  if  one  is  the  rotated  version  of  the  other. 

From  this  definition  of  the  mapping  h,  we  can  now  easily  see  the  origin  the  property  of 
connectivity  conservation  mentioned  in  the  Introduction.  If  two  nodes  Pj  and  P2  are  con¬ 
nected  on  the  spherical  mesh,  then  the  two  corresponding  nodes  Mi=hiPj)  and  M2=h{P2) 
on  the  object  mesh  are  also  connected  by  an  arc  of  the  object  mesh.  'The  property  holds 
because  of  the  definition  of  h  which  depends  only  on  the  topology  of  the  mesh,  not  on  the 
positions  of  the  nodes. 

Another  way  to  look  at  these  properties  of  SAIs  is  in  terms  of  unicity  of  representation. 
A  given  SAI  defines  a  mesh  size  and  a  distribution  of  simplex  angles.  The  unicity  property 
is  that  an  SAI  represents  a  unique  object  mesh  up  to  rotation,  translation,  and  scale.  The 
unicity  property  holds  even  in  the  case  of  arbitrary  non-convex  objects  because  of  the  con¬ 
nectivity  conservation  property.  In  fact,  we  will  show  in  Section  4.4  that  the  object  can  be 
explicitly  reconstructed  from  its  SAI. 

3.6  Alternate  Definition  of  Simplex  Angle 

Other  definitions  of  the  simplex  angle  are  possible.  In  particular,  a  conunon  definition  of 
a  discrete  curvature  index  is  the  angle  defin^  as  [1]; 

K  =  2it-  (0j-»-02  +  03) 

In  this  definition,  the  0,s  are  the  angles  between  the  vectors  joining  the  center  point  at 
which  K  is  calculated  and  its  three  neighbors  (Hgure  12).  To  be  consistent  with  cp,  the  sign 
of  K  is  positive  for  a  convex  configuration  in  which  the  center  point  is  in  the  positive  side 
of  the  plane  defined  by  its  three  neighbors,  and  negative  for  a  concave  configuration. 


Figure  12  Definition  of  Aitemate  Curvature  Angie  K 

The  behaviors  of  k  and  9  as  local  shape  varies  are  qualitatively  the  same.  As  an  example, 
let  us  consider  the  configuration  of  Figure  13  in  which  three  points  form  a  right  triangle 
with  die  lengths  of  two  of  the  sides  being  equal  to  one,  and  in  which  a  point  is  free  to 
move  on  a  line  orthogonal  to  the  plane  of  the  three  points.  We  denote  by  D  the  signed  dis¬ 
tance  between  the  point  and  the  plane.  Hgure  14  shows  the  change  in  ic  and  9  as  Z)  varies. 
In  this  figure,  9  is  scaled  to  be  between  -2n  and  +2ic  to  be  consistent  with  k.  As  expected, 
the  shapes  of  the  two  curves  are  similar:  both  angles  are  close  to  zero  as  D  vanishes,  which 
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craresponds  to  a  locally  flat  surface;  and  they  increase  monotonically  as  D  increases, 
which  corresponds  to  increasing  local  curvatures  of  the  surface.  The  graphs  show  that  k 
and  q>  are  essentially  the  same  for  large  values  of  D.  The  main  difference  between  the  two 
occurs  near  the  origin.  To  iUustrate  this  difference,  Figure  IS  shows  the  graphs  in  the 
neighbortiood  of  the  origin.  The  difference  is  now  apparent:  k  has  a  zero  tangent  at  the  ori¬ 
gin,  while  the  9  graph  exhibits  a  large  slope  at  the  origin.  In  practice,  this  means  that  k 
cannot  discriminate  well  between  local  configurations  that  correspond  to  relatively  small 
values  of  D  since  k  varies  very  slowly.  By  contrast,  because  of  the  slope,  different  local 
ccmfigurations  yield  very  different  values  of  9,  even  near  the  origin.  In  practice,  most 
shapes  have  local  configurations  located  toward  the  center  part  of  the  graph  where  the  dif¬ 
ference  between  the  two  angles  is  most  pronounced.  High  values  of  k  and  9,  where  the 
two  angles  are  essentially  the  same,  occur  mostly  at  isolated  points  of  the  surface.  This  is 
the  main  reason  for  selecting  9  over  k  to  build  our  representation  because  we  are  inter¬ 
ested  in  using  the  curvature  index  that  best  discriminates  among  shapes.  From  a  computa¬ 
tional  standpoint,  both  angles  require  essentially  the  same  amount  of  computation  even 
though  the  geometric  definition  of  k  is  simpler. 


Figure  13  Reference  Configuration  for  the  Graphs  of  Figure  14  and  Figure  15 
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Figure  14  Values  of  9  and  IC  as  Functions  of  Distance  to  Reference  Plane 
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Figure  15  Values  of  (pand  K  as  Functions  of  Distance  to  Reference  Plane  Near  the  Origin 


4  Building  Intrinsic  Representations  from  3-D  Data 

In  the  previous  sections,  we  have  defined  the  notion  of  locally  regular  mesh  and  its  asso¬ 
ciated  SAI.  In  this  section,  we  describe  the  algorithm  developed  for  computing  such  a 
mesh  from  an  input  object  We  assume  that  we  have  some  input  description  of  an  object. 
The  only  requirement  is  that  the  input  description  allows  for  computing  the  distance 
between  an  arbitrary  point  in  space  and  the  surface  of  the  object  Therefore,  input  object 
representations  such  as  polynomial  surftK^e  patches  and  polyhedra  from  CAD  models,  or 
arbitrary  triangulations  of  the  surface  are  acceptable.  Even  unstructured  sets  of  points 
from  raw  range  data  can  be  used  provided  that  the  density  of  points  is  high  enough  that  the 
distance  to  the  surface  can  be  estimated  reasonably  accurately. 

The  general  approach  is  to  first  define  an  initial  mesh  near  the  object  with  the  topology  of 
Section  3.2  and  to  slowly  deform  it  by  moving  its  nodes  until  the  mesh  satisfies  two  condi¬ 
tions;  It  must  be  close  to  the  input  object,  and  it  must  satisfy  the  local  regularity  condition. 
The  first  condition  ensures  that  the  resulting  mesh  is  a  good  approximation  of  the  object, 
while  the  second  condition  ensures  that  a  valid  SAI  can  be  derived  from  the  mesh.  Section 
4.1  describes  the  basic  algorithm  for  deforming  the  mesh;  Section  4.2  describes  the  con¬ 
struction  of  the  mesh  used  to  initiate  the  deformation  algorithm.  Section  4.3  describes  the 
algorithm  for  converting  the  final  mesh  to  a  spherical  representation  and  gives  examples 
of  building  meshes  and  SAIs  from  range  data  and  from  CAD  models.  Hnally,  Section  4.4 
shows  how  the  same  algorithm  can  be  used  to  perform  the  inverse  operation,  that  is, 
reconstructing  an  object  from  a  given  SAI. 
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4.1  Mesh  Deformation 


The  problem  is  now  to  deform  the  mesh  such  that  all  the  nodes  satisfy  two  fundamental 
properties: 

•  Mesh  nodes  must  be  as  close  as  possible  to  the  original  surface. 

•  Mesh  nodes  must  satisfy  the  normal  constraints;  a  node  is  on  the  line  parallel  to  the  nor¬ 
mal  vector  of  the  plane  formed  by  its  three  neighbors  and  passing  by  the  center  of  the 
neighbors. 

These  two  conditions  ensure  that  the  mesh  is  a  good  approximation  of  the  surface  while 
guaranteeing  that  it  is  an  intrinsic  representation.  The  formalism  of  deformable  surfaces 
[8]  is  applied  to  deform  the  mesh  until  it  satisfies  these  criteria.  Specifically,  each  node  is 
subject  to  two  types  of  forces.  The  first  type  of  forces  brings  a  node  closer  to  the  input  sur¬ 
face,  while  the  second  type  forces  the  node  to  satisfy  the  normal  constraint.  Let  be  the 
fence  of  the  first  type  applied  at  a  given  node  N,  and  Fg  be  the  force  of  the  second  type  at 
the  same  node.  Node  P  is  iteratively  moved  according  to  those  forces.  If  P,+7,  i*,,  and  P^.i 
are  the  positions  of  node  P  at  three  consecutive  iterations,  the  update  rule  is  defined  as; 

^<+1  =  +  +  +  (1) 

This  expression  is  simply  the  discrete  version  of  the  fundamental  equation  describing  a 
mechaiucal  system  subject  to  two  forces  and  to  a  damping  coefficient  D.  D  must  be 
between  0  and  1  to  ensure  convergence.  As  long  as  it  is  within  these  bounds,  D  affects 
only  the  rate  of  convergence.  A  typical  value  is  D  =  05.  Theoretically,  the  combination  of 
fences  brings  the  mesh  to  a  state  such  that  •  0  and  «  0  .  In  practice,  the  iterative 
update  of  the  mesh  is  halted  when  the  relative  displacements  of  the  nodes  from  one  itera¬ 
tion  to  the  next  are  small. 

F^  is  defined  by  calculating  the  point  P^  from  the  original  surface  that  is  closest  to  the 
node,  that  is; 


F„  =  kPP^  (2) 

Where  k  is  the  spring  constant  of  the  force  which  must  be  between  0  and  1.  The  effect  of 
the  force  is  negligible  if  the  node  is  already  very  close  to  the  surface.  Conversely,  the  force 
pulls  nodes  that  are  far  from  the  surface,  the  strength  of  the  force  increasing  with  distance. 
When  the  points  are  far  away,  it  is  desirable  to  limit  the  strength  of  the  force  to  avoid 
unstable  situations  in  which  a  node  would  move  toward  the  surface  too  quickly  and  over¬ 
shoot  the  optimal  position  by  a  large  distance.  In  practice,  k  varies  between  0.01  at  the 
beginning  of  the  iterations  to  0.4  at  the  end  of  the  iterations,  that  is,  when  the  nodes  of  the 
mesh  have  reached  a  stable  position. 

The  curvature  force  Fg  is  calculated  by  computing  the  point  Pg  that  is  on  the  line  normal 
to  the  triangle  formed  by  the  three  neighbors  of  P  and  containing  G  (Figure  9),  and  such 
that  the  mesh  curvature  at  F  and  Pg  arc  the  same:  g(F  )  =  g{P).  Those  two  conditions 
ensure  that  Pg  satisfies  the  local  regularity  condition  whue  keeping  the  original  mesh  cur¬ 
vature.  Fg  is  defined  as  a  spring  force  proportional  to  the  distance  between  P  and  Pg. 
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(3) 


F,  =  aPF, 

To  avoid  unstable  behavior  of  the  system,  the  spring  constant  a  should  be  between  0  and 
1/2.  In  practice,  a  -  i/2. 


4.2  Initialization 

Fot  the  iterative  mesh  update  to  converge,  the  mesh  must  be  initialized  to  some  shape 
that  is  close  to  the  initial  shape.  We  use  two  different  approaches  depending  on  whether 
the  input  data  is  a  set  of  data  measured  on  the  object  by  a  sensor,  or  a  synthetic  CAD 
model. 

In  the  case  of  sensor  data,  we  use  the  techniques  presented  in  [8]  using  deformable  sur¬ 
faces  to  build  a  triangulated  mesh  that  approximates  the  object  The  deformable  surface 
algorithm  fits  a  discrete  surface  to  the  input  data,  interpolating  over  the  unknown  regions, 
retaining  salient  features  of  the  surface,  if  any,  and  smoothing  the  input  data.  When  the 
representation  is  to  be  computed  from  sensor  data,  this  technique  is  particularly  effective 
because  the  deformable  surface  algorithm  tends  to  filter  out  noise  in  the  data.  This  algo¬ 
rithm  is  also  effective  in  performing  segmentation  by  separating  an  object  fiom  its  sur¬ 
roundings  in  a  complex  scene.  Once  a  triangulation  is  obtained,  the  mesh  is  initialized  by 
tessellating  the  ellipsoid  of  inertia  of  the  input  shape.  The  ellipsoid  of  inertia  is  easily  com¬ 
puted  firom  the  input  surface,  while  the  tessellation  is  computed  by  deforming  a  sphere  tes¬ 
sellated  using  the  topology  defined  in  Section  3.2.  Although  the  ellipsoid  is  only  a  crude 
approximation  of  the  object,  it  is  close  enough  for  the  mesh  deformation  process  to  con¬ 
verge.  The  distance  between  a  node  and  the  triangulated  surface  is  computed  by  finding 
the  closest  vertex  of  the  triangulation  and  by  computing  the  minimum  distance  from  the 
mesh  node  to  the  set  of  triangles  around  the  vertex. 

In  the  case  of  a  synthetic  CAD  model  as  input,  for  example  a  polyhedron,  the  ellipsoid  of 
inertia  is  computol  directly  from  the  synthetic  model.  A  regular  mesh  is  mapped  on  the 
ellipsoid  in  the  same  manner  as  in  the  previous  case.  In  this  case,  the  intermediate  repre¬ 
sentation  using  the  deformable  surface  algorithm  is  not  necessary  since  there  is  no  noise  to 
filter  out  In  fact  using  an  intermediate  model  would  degrade  the  model  by  smoothing  out 
comers  and  eliminating  high  curvature  features. 

Once  the  initial  ellipsoid  is  generated,  the  mesh  generation  is  completely  independent  of 
the  actual  format  of  the  input  data.  In  particular,  the  mesh  generation  algorithm  can  handle 
a  variety  of  representations  as  input  including  triangulations,  curved  or  polyhedral  CAD 
models,  and  sets  of  data  points  fiom  range  images.  The  only  operation  that  is  data-depen- 
dent  is  the  computation  of  the  object  point  closest  to  a  given  node. 


4 J  From  Mesh  to  SAI 

Once  a  regular  mesh  is  created  from  the  input  data,  a  reference  mesh  with  the  same  num¬ 
ber  of  nodes  is  created  on  the  unit  sphere.  The  value  of  the  angle  at  each  node  of  the  mesh 
is  stored  in  the  ccnresponding  node  of  the  sphere. 
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The  sequence  of  operations  from  input  surface  description  to  SAI  is  summarized  in  Fig¬ 
ure  16.  Ihe  SAI  building  algorithm  is  illustrated  in  Figure  17  with  range  data  as  input  and 
in  Hgtue  18  with  a  polyhedral  model  as  input.  Figure  17  (a)  shows  three  views  of  a  green 
pepper  fix>m  which  three  240x256  range  images  were  taken  using  the  OGIS  range  finder. 
The  images  are  mei^g^  and  an  initial  description  of  the  object  is  produced  using  the 
deformable  surface  algorithm.  Figure  17  (b)  and  Figure  17  (c)  show  the  initial  mesh 
mapped  on  the  ellipsoid  and  the  mesh  at  an  intermediate  stage.  Figure  17  (d)  shows  the 
final  tegular  mesh  on  the  object  Hgure  17  (e)  shows  the  corresponding  SAI.  The  meshes 
are  displayed  as  depth-cued  wireftames.The  SAI  is  displayed  by  placing  each  node  of  the 
sphere  at  a  distance  from  the  origin  that  is  proportional  to  the  angle  stored  at  that  node. 
Figure  18  (a)  to  Figure  18  (d)  show  the  same  sequence  in  the  case  of  an  object  initially 
described  as  a  polyhedron  as  generated  by  the  VAOTAGE  CAD  system.  However,  the  ini¬ 
tial  surface  is  computed  using  the  faces  of  the  CAD  model  rather  than  a  set  of  data  points 
as  in  Figure  17.  Once  this  intermediate  representation  is  generated,  the  mesh  deformation 
and  SAI  generation  algorithms  proceed  in  the  same  manner.  The  arrow  between  Figure  18 
(c)  and  Figure  18  (d)  shows  the  correspondence  between  object  mesh  and  its  SAI.  The  ver¬ 
tical  crease  in  the  middle  of  the  SAI  corresponds  to  the  concave  region  between  the  two 
cylinders.  The  top  and  bottom  regions  of  the  SAI  exhibit  large  values  of  the  angle  corre¬ 
sponding  to  the  transition  between  the  cylindrical  and  planar  faces  at  both  extremities  of 
the  object.  In  this  example,  the  SAI  exhibits  some  noise  in  regions  that  are  near  the  edges 
between  faces  of  the  object  In  practice,  the  SAI  is  smoothed  before  being  used  for  recog¬ 
nition. 


S 

final  mesh  iZ 


®  «■ 
eo  b 
e  V 


S'* 

e  B 

E  o 


-SAI 


Figure  16  Summary  of  Building  SAI  from  Input  Object  Description 
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(a)  Three  Views  of  an  Object 
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(b)  Initial  EUipsoid 


(c)  Mesh  After  10  Iterations 
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(d)  Final  Mesh 
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Figure  17  Building  SAI  flrtmi  Range  Data 
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(a)  Input  Object  Description 


(b)  Initial  Ellipsoid 


(c)  Mesh  After  10  Iterations 


(d)  Final  Mesh 
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Figure  18  Building  the  SAI  from  a  POiyhedral  Modd 


4.4  Reconstructing  Shape  from  SAI 

A  fundamental  property  of  the  SAI  representation  is  that  the  original  shape  can  be  recon¬ 
structed  fix)m  its  SAI  up  to  a  rigid  transformation  and  a  scale  factor.  In  fact,  the  same  algo¬ 
rithm  that  is  used  for  building  the  mesh  from  an  input  surface  can  be  used  to  perform  the 
inverse  operation.  Starting  with  the  standard  regular  mesh  on  the  sphere,  iteratively  apply 
the  deformation  given  by  (1)  until  the  mesh  settles  in  a  stable  configuration.  There  are  two 
differences  between  the  inverse  and  the  direct  algorithm.  First,  the  point  of  (3)  is 
defined  as  the  point  on  the  line  normal  to  the  triangle  formed  by  the  neighbors  mat  has  the 
same  angle  as  the  angle  stored  in  the  SAI,  whereas  in  the  direct  algorithm  it  is  defined  as 
the  point  that  has  the  same  angle  as  the  mesh.  Second,  F^  =  0  since  there  is  no  reference 
surface  to  attract  the  node. 


5  Matching  Objects 

We  now  address  the  matching  problem:  Given  two  SAIs,  determine  whether  they  corre¬ 
spond  to  the  same  object  If  so,  ^d  the  rigid  transformation  between  the  two  instances  of 
the  object  As  discussed  in  Section  3,  the  representations  of  a  single  object  with  respect  to 
two  different  reference  frames  are  related  by  a  rotation  of  the  underlying  sphere.  There¬ 
fore,  the  most  straightforward  approach  is  to  compute  a  distance  measure  between  me 
SAIs  for  every  possible  rotation.  Once  the  rotation  yielding  minimum  distance  is  deter¬ 
mined,  the  full  3*D  transformations  can  be  determined.  Because  it  requires  the  testing  of 
the  entire  2-D  space  of  rotations,  it  is  expensive.  We  discuss  strategies  to  reduce  the  search 
space  in  Section  S.3.  Before  that,  in  Sections  5.1  and  5.2,  we  discuss  the  distance  measure 
and  the  computation  of  the  final  rigid  transformation,  respectively. 

5.1  Finding  the  Best  Rotation 

Let  S  and  5'  be  the  spherical  representations  of  two  objects.  Denoting  by  giP),  resp. 
g’(JP),  the  value  of  the  simplex  angle  at  a  node  P  of  5,  resp.  P  of  5  \S  and  S'  are  representa¬ 
tions  of  the  same  object  if  there  exists  a  rotation  R  such  that: 

g'iP)  =  g{RP)  (4) 

For  every  point  F  of  S'.  Since  the  SAI  is  discrete,  g{RP)  is  not  defined  because  in  general 
RP  will  fall  between  nodes  of  S'.  We  define  a  discrete  approximation  of  g{RP),  G(RP),  as 
follows:  Let  F;,  P2,  Fj,  and  P4  be  the  four  nodes  of  S'  nearest  to  RP.  G(RP)  is  the 
weighted  sum  of  the  values  g(Fj).  Formally: 

4 

G{RP)  =  £W(||FF-F.||)g(Fp  (5) 

l 


24 


Where  W(d)  is  a  weighting  function  that  is  1  if  d  =  0  and  0  if  d  is  greater  than  the  average 
distance  between  nodes.  This  definition  of  G  amounts  to  computing  an  interpolated  value 
of  g  using  the  four  nearest  nodes. 

The  problem  now  is  to  find  this  rotation  using  the  discrete  representation  of  5  and  S’. 
This  is  done  by  defining  a  distance  D(S,  S’,  R)  between  SAIs  as  the  sum  of  squared  differ¬ 
ences  between  the  simplex  angles  at  the  nodes  of  one  of  the  sphere  and  at  the  nodes  of  the 
rotated  sphere.  Formally,  the  distance  is  defined  as: 

D(S.  S\R)  =  '£  ig'iP)- GiRP))  ^  (6) 

s 

The  minimum  of  D  corresponds  to  the  best  rotation  that  brings  S  and  S’  in  correspon¬ 
dence.  The  simplest  strategy  is  to  sample  the  space  of  all  possible  rotations,  represented  by 
their  angles  (<p,0,V),  and  to  evaluate  D  for  each  sample  value  ((p,-,6,-,\jr,).  The  convention 
used  for  the  rotation  angles  is  shown  in  Figure  19:  6  is  the  rotation  about  the  X  axis,  rp  is 
the  rotation  about  the  Z  axis,  and  vp  is  the  rotation  about  the  new  Z  axis.  This  approach  is 
obviously  expensive;  Section  5.3  presents  better  strategies. 

It  is  important  to  note  that  the  rotation  is  not  the  rotation  jetween  the  original  objects;  it 
is  the  rotation  of  the  representations.  An  additional  step  is  needed  to  compute  the  actual 
transformation  between  objects  as  described  below. 
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5.2  Computing  the  Full  Transformation 

The  last  step  in  matching  objects  is  to  derive  the  transformation  between  the  actual 
objects,  given  the  rotation  between  their  SAIs.  The  rotational  part  of  the  transformation  is 
denoted  by  R^,  the  translational  part  by  T^.  Given  a  SAI  rotation  R,  for  each  node  P*  of  S’ 
we  compute  the  node  P  of  S  that  is  nearest  to  RP\  Let  M,  resp.  M\  be  the  point  on  the 
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object  corresponding  to  the  node  P  of  5,  resp.  P*.  A  first  estimate  of  the  transformation  is 
computed  by  minimizing  the  sum  of  the  distances  between  the  points  M  of  the  first  object 
and  the  corresponding  points  of  the  second  object.  Formally,  the  expression  to 

minimize  is: 


(7) 

The  sum  in  this  expression  is  taken  over  the  set  of  all  the  nodes  of  the  mesh.  The  trans¬ 
formation  derived  by  minimizing  (7)  is  only  an  approximation  because  it  assumes  that  the 
nodes  from  the  two  meshes  correspond  exactly.  Due  to  the  discretization,  the  assumption 
is  not  true  in  general.  Furthermore,  the  fact  that  P*  is  the  node  nearest  to  P  in  the  S  AI  does 
not  necessarily  mean  that  M*  is  the  node  nearest  to  Af  on  the  object.  Therefore,  the  initial 
estimate  needs  to  be  refined  to  take  into  account  this  discretization  effect  A  more  accurate 
criterion  would  be  to  require  that  each  transformed  node  be  as  close  as  possible 

to  the  plane  defined  by  the  point  M  and  the  estimated  non^  vector  N  at  M.  This  defini¬ 
tion  is  more  liberal  in  that  it  does  not  require  R(^*+Tq  and  M  to  correspond  exactly. 
Instead,  the  definition  requires  R^^’+Tq  to  be  near  the  tangent  plane  at  Af.  Denoting  by 
(J?7,  T;)  the  new  estimate  of  the  transformation,  this  definition  amounts  to  finding  the  min¬ 
imum  of  the  fimction: 


(/fiAf-H7’i-Af))2  (8) 

An  iterative  technique  is  used  to  find  the  minimum  of  £/  using  {RoJ'o)  as  a  starting 
point. 

5.3  Reducing  the  Search  Space 

As  mentioned  in  Section  5.1,  the  brute  force  approach  to  finding  the  best  mesh  rotation  is 
too  expensive  to  be  practical.  However,  several  strategies  can  be  used  to  make  it  more  effi¬ 
cient  The  first  strategy  is  to  use  a  coarse-to-fine  approach  to  locating  the  minimum  of  the 
function  D  of  (6).  In  this  approach,  the  space  of  possible  rotations,  defined  by  three  angles 
of  rotation  about  the  three  axis,  (9,9,\|r),  is  searched  using  large  angular  steps  (A(p,  A6, 
Ay).  After  this  initial  coarse  search,  a  small  number  of  locations  are  identified  around 
which  the  minimum  may  occur.  The  space  of  rotations  is  again  searched  around  each 
potential  minimum  found  at  the  coarse  level  using  smaller  angular  steps  (5<p,  56, 5y).  Typ¬ 
ical  values  are  A(p=  A0=  Ay=  10°,  corresponding  to  a  36x18x36  search  space  at  the  coarse 
level.  The  rotation  space  is  then  searched  in  an  18°  wide  interval  around  each  potential 
minimum  found  at  the  coarse  level.  More  levels  of  search  may  be  more  efficient  although 
we  have  not  yet  tried  to  determine  the  best  combination  of  coarse-to-fine  levels. 

The  second  approach  is  to  use  a-priori  knowledge  about  the  relative  poses  of  the  objects 
to  reduce  the  search  space.  For  example,  the  rotation  defined  by  the  axis  of  inertia  of  the 
SAIs  can  be  used  as  a  starting  point  for  the  search.  In  general,  a  coarse-to-fine  approach 
should  be  used  in  a  relatively  large  region  centered  at  the  rotation  calculated  from  the  axis 
of  inertia.  The  use  of  a  large  region  is  necessary  because  parts  of  the  objects  may  be 
occluded,  leading  to  variations  in  the  axis  of  inertia,  and  because  the  rotation  computed 


26 


finom  the  axis  of  inertia  is  a  crude  approximation  of  the  true  rotation.  In  practice,  using  the 
axis  of  inertia  is  very  effective  in  pruning  the  search  space  as  long  as  the  visible  part  of  the 
object  is  large  enough. 

Finally,  features  on  the  spherical  representation  itself  can  be  used  to  further  narrow  the 
search.  Specifically,  local  maxima  of  g  on  both  spherical  representations  can  be  matched 
to  coiiq)ute  an  initial  rotation  to  narrow  the  search  space.  This  initial  rotation  would  be 
more  accurate  than  the  one  from  the  axis  of  inertia.  However,  it  would  be  computable  only 
if  the  spherical  representadons  exhibit  well-defined  local  maxima.  Although  we  did  not 
implement  this  strategy,  it  seems  to  be  more  appropriate  to  the  case  in  which  a  smaller  por¬ 
tion  of  the  object  is  actually  visible. 

5.4  Example 

Figure  20  shows  three  views  of  the  same  object  as  in  Figure  17  placed  in  a  different  ori¬ 
entation.  A  model  is  built  from  the  three  corresponding  range  images  using  the  approach 
described  in  4.3.  Figure  21  illustrates  the  difference  of  pose  between  the  two  models  com¬ 
puted  from  the  two  sets  of  images.  Figure  21  (a)  (resp.  Figure  21  (b))  shows  the  superim¬ 
position  of  the  cross  sections  of  the  two  models  in  the  plane  YZ  (resp.  XY).  Figure  22 
shows  the  value  of  the  S  AI  distance  measure.  The  distance  measure  is  displayed  as  a  func¬ 
tion  of  9  and  6  only  since  the  distance  is  a  function  of  three  angle  that  cannot  be  displayed 
easily.  The  displayed  value  at  (<p,6)  is  the  minimum  value  found  for  all  the  possible  values 
of  y.  The  resolution  of  the  graph  is  10°  in  both  (p  and  6,  and  the  angles  are  defined  using 
the  convention  of  Figure  19.  This  display  shows  that  there  is  a  single  sharp  minimum  cor¬ 
responding  to  the  rotation  that  brings  the  SAI  in  correspondence.  Figure  23  (a)  and  (b) 
show  the  superimposition  of  the  cross-sections  of  both  models  after  the  second  was  trans¬ 
formed  using  the  transformation  computed  from  the  SAI  correspondence  using  the  algo¬ 
rithms  of  Section  5.2.  Figure  24  shows  one  of  the  models  backprojected  in  the  image  of 
the  other  using  the  computed  transformation.  Figure  24  (a)  is  the  original  image;  Figure  24 
(b)  is  the  backprojected  model.  These  displays  show  that  the  transformation  is  correctly 
computed  in  that  the  average  distance  between  the  two  models  after  transformation  is  on 
the  order  of  the  accuracy  of  the  range  sensor.  This  example  demonstrates  the  use  of  SAI 
matching  in  the  case  of  complete  models.  In  the  next  section,  we  address  the  problem  of 
dealing  with  partial  views. 
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F^nre  20  Three  Views  of  the  Object  of  Figure  17  in  a  Different  Orientation 


(a)  Cross-Section  in  X  (b)  Cross-Section  in  Z 

F^ore  21  Relative  Positions  of  the  Models  Before  Matching 


Figure  22  Graph  of  Dbtance  Between  SAIs  as  a  Function  of  (p  and  9 
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(a)  Cross-Sectkw  in  X  (b)  Cross>Sectioa  in  Z 


Figure  23  Relative  Positions  of  the  Models  after  Matching 


(a)  Image  of  First  Model  (h)  Second  Model  Displayed  Using  Computed  Pose 


Figure  24  Display  of  the  Model  in  the  Computed  Pose 


6  Partial  Views  and  Occlusion 

Up  to  now  we  have  assumed  that  we  have  a  complete  model  of  the  object,  as  in  Figure 
18,  or  that  we  have  data  covering  the  entire  surface  of  the  object,  as  in  Figure  17.  This 
assumption  is  appropriate  for  building  reference  models  of  objects.  During  the  recognition 
phase,  however,  only  a  portion  of  the  object  is  visible  in  the  scene.  The  matching  algo¬ 
rithm  of  Section  5  must  be  modified  to  ^low  for  partial  representations.  The  algorithm 
used  fcHT  extracting  the  initial  surface  model  is  able  to  distinguish  between  regions  of  the 
mesh  that  are  close  to  input  surfaces  or  to  data  points,  and  parts  that  are  interpolated 
between  input  data.  The  tint  type  of  region  is  the  visible  part  of  the  mesh,  and  the  second 
type  is  the  occluded  part  of  the  mesh.  Therefore,  even  though  the  representation  is  always 
a  mesh  mapped  on  a  closed  surface,  it  is  always  possible  to  determine  which  nodes  of  the 
mesh  represent  valid  data. 
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The  situation  is  illustrated  in  Figure  25  in  the  case  of  a  two  dimensional  contour.  In  Fig¬ 
ure  25  (a)  a  contour  is  approximated  by  a  mesh  of  eight  points.  The  mesh  is  assumed  to  be 
regular,  that  is  all  the  points  of  the  mesh  are  equidistant.  Let  L  =  8/  be  the  total  length  of 
the  mesh.  Figure  25  (b)  shows  the  same  contour  with  one  portion  hidden.  The  occluded 
portion  is  shown  as  a  shaded  curve.  The  visible  section  is  2q)proximated  by  a  regular  mesh 
of  eight  nodes  of  length  Lj  =  8/j.  Since  the  occluded  part  is  interpolated  as  a  straight  line, 
the  length  of  this  mesh  is  smaller  than  the  total  length  of  the  mesh  on  the  original  object:  L 
>  Lj.  Conversely,  the  length  of  the  part  of  the  representation  corresponding  to  the  visible 
part,  1/2  shown  in  Figure  25  (d),  is  greater  than  the  length  of  the  same  section  of  the  curve 
on  the  original  representation,  L*  shown  in  Figure  25  (c).  In  order  to  compute  the  distance 
measure  of  Section  5,  the  SAI  of  the  observed  curve  must  be  scaled  so  that  it  occupies  the 
same  length  on  the  unique  circle  as  in  the  reference  representation  of  the  object.  If  L*  were 
known,  the  scale  factor  would  be: 


k  =  ^  (9) 

In  reality,  L*  is  not  known  because  we  do  not  yet  know  which  part  of  the  reference  curve 
corresponds  to  the  visible  part  of  the  observed  curve.  To  eliminate  L*,  we  use  the  relation: 


(10) 


This  relation  simply  expresses  the  fact  that  the  ratios  of  visible  and  total  length  in  object 
and  representation  spaces  are  the  same,  which  is  always  true  when  the  mesh  is  regular. 
Since  the  left-hand  side  involves  only  known  quantities,  total  length  of  nxxlel  and 
observed  visible  length,  L*  can  be  eliminated  by  combining  (9)  and  (10): 


.  2k 


(11) 


The  situation  is  the  same  in  three  dimension  except  that  lengths  are  replaced  by  areas  A, 
A j,  A2,  A*,  with  obvious  notations.  Relation  (11)  becomes: 


4kAi 
*  A2A 


(12) 


The  direct  extension  from  two  to  three  dimension  is  only  an  approximation  because  the 
equivalent  of  relation  (lO),Aj/A  =  AIAk^  holds  only  if  the  area  per  node  is  constant  over 
the  entire  mesh.  In  practice,  however,  the  area  per  node  is  nearly  constant  for  a  mesh  that 
satisfies  the  local  regularity  condition. 

Once  k  is  computed,  the  appropriate  scaling  needs  to  be  applied  to  the  SAI.  The  scaling 
algorithm  is  illustrated  in  Figure  26:  if  C  is  the  center  of  the  visible  region  on  the  represen¬ 
tation  sphere,  a  node  P  such  that  9  is  the  angle  {OP,  OQ  is  moved  to  the  point  P*  on  the 
great  circle  that  contains  P  and  C  such  that: 

1-COS0'  =  k(l-cos0)  (13) 
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where  0’  is  the  angle  {OP\  OC)  and  it  is  the  scale  factor.  This  mapping  is  chosen  because 
it  guarantees  that  the  area  of  the  visible  region  is  scaled  exacdy  by  k  if  the  region  is  circu¬ 
lar.  Even  if  the  region  is  not  circular,  the  mapping  is  a  reasonable  approximation. 


(c)SAI  of  Complete  Object  (d)SAl  of  Partial  View 

Figure  25  Matching  Partial  Representation  in  Two  Dimensions 


Figure  26  SAl  Scaling  Algorithm 

The  key  in  this  algorithm  is  the  connectivity  conservation  property  of  the  S AI  mentioned 
previously.  Specifically,  if  a  connected  patch  of  the  surface  is  visible,  then  its  correspond¬ 
ing  image  on  the  SAI  is  also  a  connected  patch  on  the  sphere.  This  property  allows  us  to 
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bring  the  two  connected  patches  into  correspondence  using  a  simple  scaling.  Establishing 
the  correspondence  is  not  possible  in  the  case  of  the  EGI  representation,  in  which  the 
^helical  representations  of  an  object  and  of  a  patch  on  the  object  may  be  completely  dif¬ 
ferent  If  the  object  is  represented  by  an  implicit  equation,  e.g.,  algebraic  function  or 
supeiquadrics,  then  the  coefficients  of  the  equation  computed  from  the  entire  object  sur¬ 
face  may  be  completely  different  from  the  ones  computed  from  only  a  patch  on  the  object 
surface. 

We  now  show  two  examples  of  recognition  in  the  presence  of  occlusion.  In  the  first 
example,  a  range  image  of  an  isolated  object  is  taken.  Hien  a  complete  model  of  the  object 
is  matched  with  the  SAI  representation  from  range  data.  Figure  27  shows  the  intensity 
image  of  the  object  Only  about  30%  of  the  object  is  visible  in  the  image.  The  remaining 
70%  of  the  representation  built  from  the  image  is  interpolated  and  is  ignored  in  the  estima¬ 
tion  of  the  SAI  distance.  Figure  28  (a)  shows  the  set  of  three  registered  views  used  to  build 
the  reference  model,  and  Figure  28  (b)  shows  the  SAI  of  the  reference  model  used  for 
matching.  Figure  29  shows  the  superimposition  of  scene  points  and  reference  model  after 
transformation.  Figure  30  displays  the  graph  of  the  distance  between  SAIs  as  function  of 
rotation  angles.  Figure  30  (a)  shows  two  views  of  the  distance  as  a  function  of  tp  and  6. 
Figure  30  shows  the  same  function  displayed  in  <p-y  space.  These  displays  demon¬ 
strate  that  there  is  a  well-defined  minimum  at  the  optimal  rotation  of  the  SAIs.  Figure  31 
shows  the  model  backprojected  in  the  observed  image  using  the  computed  transformation. 
In  this  example,  the  reference  model  was  computed  by  taking  three  registered  range 
images  of  the  object  as  in  the  example  of  Figure  17. 

In  the  second  example,  the  reference  model  is  the  CAD  model  of  Figure  18.  The 
observed  scene  is  shown  in  Figure  32.  The  result  of  the  matching  is  shown  in  Figure  33 
and  Figure  34.  Only  part  of  the  object  is  visible  in  the  image  because  about  of  self  occlu¬ 
sion  and  because  of  occlusion  from  other  objects  in  the  scene. 

In  both  examples,  the  deformable  surface  algorithm  is  used  to  separate  the  object  from 
the  rest  of  the  scene  and  to  build  an  initial  surface  model.  If  there  is  no  data  point  in  its 
vicinity,  the  visible  portion  of  the  object  mesh  and  the  corresponding  SAI  are  identified  by 
marking  a  node  of  the  mesh  as  interpolated.  Using  the  algorithms  presented  in  this  section, 
the  SAI  of  the  observed  object  was  scaled  based  on  the  size  of  the  visible  area.  As  an 
example.  Figure  35  (a)  shows  the  SAI  computed  from  the  image  of  Figure  27,  Figure  35 
(b)  shows  the  SAI  after  the  scaling  in  applied  to  compensate  for  occlusions.  The  density  of 
points  increases  in  the  region  that  corresponds  to  the  visible  part  of  the  object  (indicated 
by  the  solid  arrow).  Conversely  the  density  of  points  decreases  in  the  region  correspond¬ 
ing  to  the  occluded  part  of  the  object  (in^cated  by  the  shaded  arrow).  These  examples 
show  that  the  SAI  matching  algorithm  can  deal  with  occlusions  and  partial  views,  even 
when  only  a  relatively  small  percentage  of  the  surface  is  visible. 
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Figure  27  Input  Inuge 


Figure  28  Reference  Model 


Figure  29  Crass  Sections  of  Registered  Model  and  Scene  Using  the  Image  of  L  igure  27  and  the  Modei 

from  Figure  28 
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Figure  30  Sum  of  Squared  DifTereuces  of  SAIs  as  Function  of  Rotation  Angles 


Figure  32  Input  Image 


Figure  33  Cross  Sections  of  Registered  Modei  and  Scene  Using  the  Image  of  Figure  32  and  the  Model 

of  Figure  18 
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(a)  Input  Image  (b)  Model  after  lYansformatioD 

Figure  34  Display  of  Model  Using  the  Pose  Computed  flrom  the  Matching 


(a)  SAI  firom  Scene  (b)  SAI  after  Scaling 

Figure  35  Effect  of  Occlusion-Compensating  Scaling  on  SAI  of  Observed  Object 


7  Conclusion 

In  this  paper,  we  introduced  a  new  approach  for  building  and  recognizing  models  of 
curved  objects.  The  basic  representation  is  a  mesh  of  nodes  on  the  surface  that  satisfies 
certain  regularity  constraints.  We  introduced  the  notion  of  simplex  angle  as  a  curvature 
indicator  stored  at  each  node  of  the  mesh.  We  showed  how  a  mesh  can  be  mapped  into  a 
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spherical  representation  in  canonical  manner,  and  how  objects  can  be  recognized  by  com¬ 
puting  the  Stance  between  spherical  representations. 

The  SAI  representation  has  many  desirable  properties  that  make  it  very  effective  as  a 
tool  for  3-D  object  recognition: 

•  The  SAI  is  invariant  with  respect  to  translation,  rotation,  and  scaling  of  the  object.  This 
is  not  true  of  most  other  commonly  used  representations.  This  invariance  allows  the 
recognition  algorithm  to  compare  shapes  through  the  computation  of  distances  between 
SAIs  without  requiring  explicit  matching  between  object  features  or  explicit  computa¬ 
tion  of  object  pose. 

•  The  SAI  preserves  connectivity  between  parts  of  the  object  in  that  nodes  that  are  neigh¬ 
bors  on  the  object  mesh  are  also  neighbors  on  the  SAI.  Thus  the  SAI  does  not  exhibit 
the  same  ambiguity  problem  for  non-convex  objects  as  the  EGI  and  CEGI  representa¬ 
tions. 

•  The  SAI  representation  can  handle  partial  views  and  occluded  objects.  The  basic 
approach  is  to  measure  the  area  of  the  visible  portion  of  an  object  observed  in  a  scene, 
and  deform  the  SAI  mesh  model  so  that  the  percentage  of  the  sphere  corresponding  to 
the  visible  area  is  the  same  in  both  model  and  scene  SAIs.  This  approach  to  recognition 
of  occluded  objects  is  practical  thanks  to  the  property  of  connectivity  conservation 
described  above.  Specifically,  a  connected  visible  region  of  an  object  corresponds  to  a 
coimected  region  on  the  corresponding  SAI. 

Results  show  that  the  SAI  representation  is  successfully  used  to  determine  the  pose  of  an 
object  in  a  range  image  including  occlusion  and  multiple  objects.  This  approach  is  partic¬ 
ularly  well  suited  for  applications  dealing  with  natu^  objects,  lypically,  conventional 
object  modeling  and  recognition  techniques  would  not  work  due  to  the  variety  and  com¬ 
plexity  of  shapes  that  may  have  to  be  handled.  The  approach  is  general  enough  that  it  can 
also  convert  manually  built  models  to  the  SAI  representation. 

Many  issues  remain  to  be  addressed.  First,  we  need  to  improve  the  search  for  the  mini¬ 
mum  distance  between  SAIs  during  the  recognition  phase.  This  improvement  can  be 
achieved  by  improving  the  coarse-to-fine  approach  to  extrema  localization,  and  by  using 
cues  computed  from  the  original  data  to  restrict  the  area  in  which  the  extrema  are 
searched.  Another  interesting  direction  of  research  is  the  parallel  implementation  of  the 
algorithm  which  should  be  possible  since  computations  at  each  node  of  the  sphere  and  for 
each  possible  rotation  are  independent  of  each  other.  A  third  issue  is  the  extension  of  those 
techruques  to  objects  that  are  not  topologically  equivalent  to  the  sphere,  such  as  torus¬ 
shaped  objects  or  open-surfaced  objects.  This  extension  requires  the  definition  of  mesh 
topology  and  global  regularity  for  those  classes  of  shapes.  To  apply  the  technology  to 
more  complex  scenes,  actional  range  image  processing  needs  to  be  developed  to  isolate 
objects  in  the  scene  from  one  another.  Currently,  the  algorithm  introduced  in  [8]  uses  a 
fixed  window  around  the  center  of  each  potential  object  in  the  scene.  A  better  initial  seg¬ 
mentation  is  needed  to  deal  with  complex  scenes.  Finally,  an  important  issue  is  to  deter¬ 
mine  the  appropriate  mesh  resolution,  as  given  by  the  number  of  nodes,  that  is  needed  for 
a  particular  tq)plication.  Currently,  the  empirical  selection  of  the  size  of  the  mesh  can  lead 
to  two  possible  problems.  If  the  size  of  the  mesh  is  too  small,  important  details  of  the 
object  may  be  undersampled.  On  the  other  hand,  if  the  mesh  is  too  large,  computation  time 
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for  both  model  building  and  recognition  becomes  prohibitive.  The  density  of  nodes  should 
be  computed  based  on  the  average  and  maximum  curvatures  of  the  surface. 
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Appendix  A:  Simplex  Angle  Computation 

Computing  the  simplex  angle  directly  from  the  definition  of  Section  3.4  is  inefficient  and 
numerically  imstable.  In  this  Appendix,  we  describe  the  method  used  in  the  implementa¬ 
tion.  We  first  present  the  algorithm  and  then  give  a  proof  that  the  algorithm  is  consistent 
with  the  definition  of  Section  3.4.  Starting  with  the  geometry  of  Figure  36,  we  define  the 
following  notations: 

•  P  is  a  node  of  the  mesh  with  neighbors  (PjJ*2J*3)‘ 

•  iV  is  the  unit  vector  normal  to  the  plane 

•  Vi  is  the  vector  PPj. 

•  Li  is  the  magnitude  of  V,-.  ^ 

•  is  the  unit  vector  parallel  to  Vj!  ~  X" 

The  vectors  Tj  and  t,-  can  be  represented  on  a  unit  sphere  of  center  O  (Figure  37  (a)). 
Geometrically,  the  vector  Ti  is  the  intersection  of  the  unit  vector  /,•  with  a  sphere  of  diame¬ 
ter  N,  the  center  of  which  is  denoted  by  O;.  Another  way  to  define  the  vectors  Ti  is  to 
observe  that  (XiJ‘2^'^3)  ^  image  by  a  spherical  inversion  of  pole  P  of  the  points 
(PjJ*2>^3)'  The  inversion  maps  the  plane  iPjJP2tPs)  onto  the  sphere  of  diameter  iV. 

The  triplet  of  points  (Tj,  T2,  Tj)  defines  a  plane  11  Let  d  be  the  distance  between  the  ori¬ 
gin  Oj  of  the  sphere  of  diameter  N  and  FI  (Figure  37  (b)).  We  compute  the  simplex  angle 
as: 

coscp  =  Id  (14) 

This  definition  gives  an  angle  between  0  and  it.  The  sign  of  the  angle  depends  on 
whether  P  is  on  the  positive  or  negative  side  of  the  plane  defined  by  its  neighbors  and  the 
normal  vector  JV  assumed  to  be  pointing  toward  the  outside  of  the  object.  This  definition  of 
(p  can  be  computed  more  efficiently  since  it  requires  only  three  dot  products  and  three  nor¬ 
malizations  to  compute  the  vectors  7,-,  and  another  dot  product  to  compute  the  distance  to 
n  instead  of  the  radii  of  a  circumscribed  sphere  and  a  circumscribed  circle  using  the  defi¬ 
nition  of  Section  3.4.  It  is  also  better  conditioned  in  that  when  the  surface  becomes  nearly 
flat,  the  vectors  7/  converge  toward  the  origin  of  the  unit  sphere  and  therefore  d  =  1/2  and 
<p  =  0.  By  contrast,  the  sphere  radius  used  in  the  previous  definition  goes  to  infinity  as  P 
becomes  close  to  the  plane  of  its  neighbors. 


Figure  34  Geometry  of  Simplex  Angle  Computation 
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(a)  Inversion  of  Unit  Direction  Vectors 


N 


(b)  Cross-Section  of  the  Sphere  of  Diameter  N 


Figure  37  Configuration  after  Spherical  Inversion 

We  now  show  that  the  angle  calculated  with  this  algorithm  is  indeed  the  same  as  the 
angle  defined  in  Section  3.4.  The  proof  will  proceed  by  expressing  simplex  angle  as  a 
function  of  the  vectors  Vj,  to  use  the  relation  between  V,-  and  T,  to  express  (p^  as  a  function 
of  vectors  Ti,  and  to  identify  the  resulting  expression  as  the  definition  used  in  the  algo¬ 
rithm  presented  above. 

The  original  definition  of  (p^  can  be  written  as: 


sintp^  =  -p  (15) 

where  is  the  radius  of  the  circle  circumscribed  to  the  triangle  (i®;,  P2,  P3),  and  r  is  the 
radius  of  the  sphere  circumscribed  to  the  tetrahedron  {Pi,  P2,  P3,  P)  (Figure  10).  With  the 
current  notations,  we  have: 


\lI  (V2  X  V3)  +  ( V3  X  V,)  +  l]  ( Vj  X  V2) 

2|V„  V2,  V3I 

o  2IIV2XV3  +  V3XV1  +  V1XV2I 


where  |Vi,  V2,  V3I  is  the  determinant  of  the  three  vectors.  Let  D  be  distance  of  P  to  the 
plane  of  its  neighbors,  of  normal  vector  N.  By  definition, 
D  =  Vj  •  =  V2  •  W  =  V3  •  iV,  a  symmetrical  way  of  writing  these  equalities  is: 


D  = 


\Vv^2^V,\ 

I V2  X  V3  +  V3  X  Vj  +  Vj  X  ^2! 


(17) 


Replacing  the  right-hand  side  of  (17)  by  D  in  the  expression  of  rjr  obtained  from  (16), 
we  get: 
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(18) 


LliV^y-V-i)  +l|(V3xVi)  +Z^(ViX  V2) 


Replacing  V,- JV  by  D  in  the  definition  of  we  get  the  relations: 


V~V  = 

‘  j 


T-T 


Substituting  (20)the  right-hand  sides  of  (19)  in  (18),  we  get  the  new  expression  for  the 
ratio  of  radii; 


%  ^  JZiiMEtldML  oo) 

r  |irjxr3+r3xr,+r,xrj 

The  right  hand  side  of  (20)  is  exactly  twice  the  radius  R  of  the  circle  circumscribed  to  the 
triangle  formed  by  the  points  T;,  T2,  and  Tj.  Therefore,  from  (15),  sintp^  is  equal  to  7R. 
Equivalently,  cos<p^  is  the  distance  between  the  origin  and  the  plane  11  (Figure  38).  There¬ 
fore  the  two  definitions  of  the  simplex  angle,  sintp  =  r^/r  and  costp  =  2d,  are  equiva¬ 
lent 
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